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Negative experiences of university mathematics education are often laid at the feet of online 
or blended learning. However, data collected as part of a five-year project at two universities 
suggests that there is much more to consider in determining the quality of preservice teacher 
mathematics education courses. This paper outlines a methodology that investigates the 
experiences of pre-service education teachers (PSTs) in relation to their journey of learning 
how to teach primary mathematics delivered via a variety of modes. Results indicated that 
the mode of delivery is not the critical factor as course design, teacher knowledge, and 
building rapport seem to be more influential in student success. 


In recent years mathematics education has been in the spotlight in Australia with the 
most recent TIMMS (2016) and PISA (2016) results for Australian students indicating a 
continuing decline in mathematical performance (when compared internationally). Poor 
scores on these tests are contributing to the current zeitgeist that university mathematics 
education courses in Australia are failing future school students. Our roles for the past five 
years has been as mathematics educators — at two different universities (Author 2), teaching 
mathematics education to 1‘ and 2" year pre-service teachers (PSTs), in a range of modes - 
fully online (Author 2), blended (face-to-face and online components) and solely face-to- 
face (F2F). The success or otherwise of university courses are often attributed to the mode 
of delivery with proponents of the various forms of delivery (online, blended or F2P) citing 
success in courses as primarily a consequence of delivery mode. In particular, negative PST 
experiences of university mathematics courses are often laid at the feet of online, and to a 
lesser extent, blended learning (Larkin, 2017). However, our experiences suggest that there 
is much more to consider in the puzzle of successful university mathematics education. This 
paper may assist other preservice mathematics educators when planning learning 
experiences for their students. 


What the literature tells us 


Concerns about mathematics education are not new; however, in some sense, a perfect 
storm impacts on the PSTs in this research who are: 1‘ or 2" year students; studying 
university courses offered largely in online or blended mode; often anxious about teaching 
mathematics; and, often returning to education after completing secondary schooling 10-20 
years earlier. Data suggest that many PSTs fail to enjoy or recognise the personal relevance 
of mathematics. Chubb (2014) writes of the broader disenfranchisement within mathematics 
education contributing to the decline in the number of students studying mathematics in 
Senior Secondary School or at university. These findings point to the need to ensure that 
PST’s graduate with high levels of mathematical content knowledge (MCK), mathematical 
pedagogical knowledge (MPK), and with positive attitudes towards mathematics. The 
development of positive attitudes towards mathematics is identified as a core requirement of 
mathematics education courses given the persistence of negative attitudes held by PSTs that 
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prove highly resistant to change (Grootenboer, 2008) and which often exhibit themselves in 
the form of anxiety. Even more problematic is that, whilst mathematics anxiety exists in 
almost all educational contexts, it appears to be much more prevalent in primary, pre-service, 
mathematics education students (Peker, 2009). 

A further dimension to consider is the increasing use of online components as part of a 
contemporary university experience. This, in part, is due to economic imperatives as it is 
often more financially viable to offer online courses to large cohorts and also an 
acknowledgement of the changed landscape for PSTs who are likely to be juggling demands 
imposed by work, family, and study. From a university’s perspective, the increased use of 
online components is rationalised as an appropriate response to the perceived needs of 
university students for “anywhere, anytime” learning. Although research findings are mixed, 
they generally indicate that the use of online lectures contributes to both cognitive and 
affective positive outcomes for students. However, some evidence suggests areas of 
challenge including infrastructure issues; technical quality of online components, and time 
management. In addition, the use of online lectures and/or online tutorials can heighten 
anxiety for students who are not “tech savvy”. A final consideration in online PST 
mathematics education is the potential “loss of relational contact” (Kim, 2011, p. 763) with 
negative impacts on PST attitudes towards mathematics. Given the considerations raised in 
the literature regarding PST mathematics education, we sought to answer the following 
question: 

What is the impact of mode of delivery (online, blended, or F2F), and course teaching 
personnel, on undergraduate PSTs experience of Primary mathematics education courses? 


Method 


As reflective educators, we use a design-experiment approach to continually improve the 
mathematics education courses we teach. In brief, a design-based experiment is concerned 
with the study of learning in specific contexts and then extending knowledge by generating 
models of successful innovation. The design-based experiment cycle of data collection and 
reflection is an authentic research approach as teaching academics are best placed to identify 
changes that need to be made to improve learning and teaching (Cohen, Manion & Morrison, 
2002). 

Although this research is not conducted as an experiment, it is worthwhile to note that 
there is a great deal of homogeneity between the four cohorts discussed in this paper and 
therefore some measure of control of some of the educational variables is possible. For 
example: both universities offered Bachelor of Primary Education Degrees registered by the 
same accrediting authority; the content of the courses remained largely consistent; each 
course was supported by an online Learning Management System (course profile, course 
readings, lecture notes, tutorial notes, additional resources); the PSTs were 1* or om year 
students and comprised a mixture of immediate school leavers and mature age students; and 
each cohort received three hours of “contact” with the teaching team each week. Provided 
below is a brief description of each of the four cohorts in the study. 

Cohort A (2011): This cohort had no physical F2F access as all course elements were 
delivered online. These students received a weekly one-hour recorded lecture (audio with 
accompanying PowerPoint slides), and a two-hour Wimba tutorial (Wimba is a proprietary 
software and provides a virtual classroom with chat, voice, interactive white board, and 
breakout rooms for small group activities). 

Cohort B (2015): This cohort had a mixture of a one-hour online lecture (with embedded 
video of the lecturer), a one-hour F2F workshop, and a one-hour F2F tutorial. This delivery 
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mode is referred to in this paper as the (1+1+1) model. Each one-hour interactive workshop 
was conducted with the entire cohort. The tutorials consisted of a maximum of 30 students 
in a classroom environment (group work at tables). 

Cohort C (2016): This cohort received all contact hours in F2F mode. The three hours 
were comprised of a one-hour lecture, one-hour workshop and one-hour tutorial. It was thus 
similar to the 2015 model with the difference being the online lecture was replaced with a 
F2F lecture that was also recorded for later access by the students. 

Cohort D (2017): This cohort is identical to Cohort B in terms of delivery mode, with 
the difference being a new lecturer (Author 1) delivering the F2F components. As in 2015, 
these students received the (1+1+1) delivery mode. 

Thus, the significant variable for three of the four cohorts was the mode of delivery of 
the ‘contact’ hours and the variable for the fourth cohort was a change in personnel 
delivering the face-to-face components. The data set for this study consists of end-of-course 
Student Evaluation of Teaching (SET) quantitative scores and qualitative feedback; online, 
in-course surveys (ICS) during two of the courses (2015; 2016); and, our reflections on the 
course delivery over the period under investigation. Each end of semester survey sought to 
gather information from students regarding their experience of the course. Although there 
have been concerns expressed in the literature regarding the reliability and validity of SET 
evaluations (Larkin, 2017; Rowan, 2013), Likert scale SET scores are used here as blunt 
indicators of course quality (Table 1). 


Table 1 
End of Course Feedback (2011-2017) and *In Course Surveys (2015 / 2016) 
Cohort Number of Number of Response Rate 
Students Responses 
Fully Online (2011) 223 124 56% 
Blended (2015) 136 53 39% 
Blended (2015)* 136 45* 33% 
Face to Face (2016) 130 31 24% 
Face to Face (2016)* 130 66* 51% 
Blended — New Lecturer (2017) 153 49 32% 


Although the criteria differ slightly between the two universities (See Tables 2 and 3), 
each end of semester survey evaluated similar aspects of the courses i.e. learning 
expectations, course structure, individual treatment, assessment feedback etc. In addition to 
the end of semester feedback, qualitative data regarding student experience of the 2015 and 
2016 cohorts were collected via anonymous, in-course surveys (ICS) conducted mid-way 
through each course. These surveys included a number of open-ended questions regarding 
the mode of lecture delivery. 

In order to make sense of the data collected in the SET and ICS, Thematic Analysis was 
used. According to Braun and Clark (2006) Thematic Analysis is a “method for identifying, 
analysing and reporting patterns (themes) within data. It minimally organizes and describes 
your data set in (rich) detail” (p.79). Broadly speaking, Thematic Analysis involves a range 
of processes (i.e. familiarisation, generating codes, and then searching for, reviewing, 
defining and communicating themes). Although the overall process appears prescriptive, it 
is important to acknowledge that the various processes are guidelines to be applied flexibly 
to each research context (e.g. in this research, the familiarization process did not involve 
transcription and we were already very familiar with the data as it is a common component 
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of our twice-yearly evaluation practices). A second observation regarding Thematic Analysis 
is that it is a recursive rather than linear process and thus, there is movement back and forth 
between the phases (e.g. we commenced with the 2011 data which was then revisited after 
we had processed the 2015 and 2016 data). 


Impact of the Different Delivery Modes 


Findings from each of the four discrete cohorts are presented below. As each course 
offering had a particular targeted modification (i.e. mode of delivery or team personnel), the 
response of PSTs to these modifications are analysed first. 


Cohort A — 2011 (Author 2 Only - Fully Online: University One) 


As can be seen from the data in Table 2, when compared with other courses, and other 
faculties, the PSTs evaluated the course as being very successful. 


Table 2: 
Student Evaluation of Teaching (Cohort A-Fully Online: University One — Identifying data 
omitted). 


Comparative Means No. . : Std 
Questions with a scale of 5, and from the SEC survey Ans Dev 


SEco1: The learning outcomes and the expected standards of this course were cleartome | 124/223 is | 3.93 | 0.78 
SEco2: Course content was presented in ways which greatly assisted my learning. 124/223 | 3.76 |0.91 

: . i 1 } 4 4 | 
SEco3: Where relevant, teaching in this course helped me to leam effectively. 124/223 y F ! 3.71 | 0.86 


Ecos: | was able to obtain individual help in this course when | needed it. 123/223 7 7 : 3.79 | 0.74 


Ecos: I have learned and understood the subject materials in this course. 123/223 s 3.99 | 0.87 


SECo6: The learning resources (library, study guides, handouts, text, web resources, etc) 
were adequate for my study in this course. 


Secor: | have learned to make connections between this subject and others. 123/223 7 : 3.97 | 0.85 
Ecos: Overall | have learned a lot in this course. 122/223 | | § | 4.00 | 0.81 
secos: The workload in this course was too high. 123/223 y 2.97 | 1.07 
sec10: The assessment tasks were appropriate to the aims of this course. 124/223 | | | . | 3.94 | 1.03 | 
SEC11: The criteria used to assess student work were clear. 123/223 | | [3.78 | 0.96 | 
SEc12: Feedback from assignments was timely. 123/223 | | | | | | 3.69 | 0.93 | 


SEC13: My understanding of the subject has improved as a result of feedback from 
assignments. 


SEC14: Overall | was satisfied with the quality of this course. 122/223 3.75 | 0.86 


123/223 7 3.86 | 0.91 


123/223 3.59 | 1.02 


The major innovation in the course was the provision of a weekly Wimba (online 
classroom) tutorial with much of the feedback (37 comments) discussing various positive 
aspects of this innovation. Sample comments included “Wimba tutorials- opportunity to feel 
part of a community of learners” {SET2011} and “Wimba classes were great — live 
opportunity to see resources and to ask questions and gain from other class members” 
{SET2011}. This innovation enabled the provision of a pedagogical space for me to connect 
synchronously with the PSTs online, where effective mathematics pedagogy could be 
demonstrated. I was therefore able to replicate many of the affordances of a F2F teaching 
environment in this virtual space. 


Cohort B — 2015 (Author I and 2 - Blended 1-1-1; University Two) 


Cohort B comprised PSTs at University Two who received very similar content to the 
earlier cohort. The major innovation in this course was the blended delivery (online lectures 
and F2F workshops and tutorials). A further modification was the inclusion of video of the 
lecturer in the online component (as opposed to just audio and PowerPoint slides in previous 
offerings of this course in 2013 and 2014). Once again, the course was evaluated favourably 
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(See Table 3) via end of semester student evaluations. Feedback from the ICS also indicated 
a positive experience suggesting that the use of video was very important for their overall 
success. Sample ICS responses report both: gains in attention - “J have noticed that I pay 
more attention to the video lectures as opposed to my zoning out on the non-video lectures. 
It’s not you, it’s me. I just learn better visually” {ICS2015}; and also a heightened sense of 
connection with me as the lecturer - “J prefer the video as it feels like Iam at a real lecture 


and it feels more personal. Without the video I feel like that, as a student, I don't really mean 
anything’ {ICS2015}. 


Table 3 
Student Evaluation of Teaching (Cohorts B, C & D: University Two). 


Descriptor Mean (2015 n= 53) Mean (2016 n= 31) Mean (2017 n=49) 
This staff member... Blended* Face to face* Blended* 


(New Lecturer) 


Presented material in an organised way 4.8 4.8 4.8 
Presented material in an interesting way 4.7 4.9 4.9 
Treated me with respect 4.8 4.9 5.0 
Showed good subject matter knowledge 4.9 4.9 5.0 
Overall I am satisfied with the teaching 4.8 4.9 4.9 


*Results are mean scores from across three campuses rounded to the nearest tenth. n = number of responses 


Cohort C — 2016 (Author I and 2 - Fully Face-to-Face: University Two) 


Cohort C comprised students at University Two and the major innovation in this course 
was the delivery of all lectures, workshops and tutorials in F2F mode. As was the case in 
each of the two previous course offerings, end of semester student feedback (Table 3) 
indicated a positive course experience. In addition, data from the 66 In-course survey 
respondents generally indicating a preference for the full F2F course experience (54 positive, 
6 neutral and 6 indicating a preference for the blended mode). Sample ICS responses 
included “The F2F lectures are more interactive and the questions people ask are often 
interesting” {ICS2016}; and “J prefer F2F as it gives you more of a chance to engage with 
the lecturer/content and to ask questions and to have them answered” {1CS2016}. One of 
the neutral PST responses noted that “Both online and F2F have benefits but I prefer the 
F2F model as I can stay motivated and keep coming to the lectures instead of missing the 
lecture videos in Mathematics One’ {ICS2016}. 


Cohort D — 2017 (Blended 1+1+1 with New Lecturer: University Two) 


Cohort D comprised PSTs at University Two who received an identical mode of delivery 
to that outlined earlier for Cohort B i.e. one-hour online lecture, one-hour F2F workshop and 
one-hour tutorial. The only difference for this course offering was that all F2F teaching, 
including weekly workshops and tutorials, was delivered by a new Lecturer (Author 1). The 
first author had previously worked on the course as a tutor and was experienced in the course 
assessment and practices in earlier course offerings. The previous lecturer (Author 2) 
remained involved in the course as course convenor. Author 2 had taken a step back from 
course teaching due to commitments to a large national research project. Despite the change 
in the teaching team, the course was still evaluated very favourably (See Table 3) via end of 
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semester student evaluations and maintained the high scores obtained in previous course 
offerings. 

Feedback from the ECS indicated that the structure and specific combination of teaching 
approaches were beneficial to student learning outcomes. With specific reference to the 
online lecturers, sample responses included — “The online lectures were structured 
brilliantly and the  workshops/tutorials are useful in complementing _ this 
information” {SET2017}. Furthermore, students felt the combination of face to face and 
online components was balanced and offered enjoyment in learning the mathematical 
content that some students find difficult to comprehend and appreciate — “The one hour 
lecture followed by a one hour workshop and concluding with an one hour tutorial provides 
a balanced learning experience where we get plenty of hands-on learning” {SET2017}; “T 
enjoyed how there was a variety of ways the content was taught- online, workshops and 
tutorials etc.” {SET2017}. An important point to note about this cohort is the emphasis in 
the end of semester feedback on the structure of the course and not on the personnel involved 
in delivering the course. 

Although each of four cohorts received a very different mode of course delivery (online, 
blended or F2F) or team personnel, the combined data indicates that the PSTs in each 
respective cohort judged each course as very successful. Thus, regardless of the mode, the 
student satisfaction scores remained well above average across the four cohorts and in the 
very high 4.7-5.0 range for Cohort B, C, & D. Clearly then, this quantitative data indicates 
that neither the mode of delivery nor the team personnel are the sole factor in the success of 
the course. Interestingly, there was little mention in the end of semester feedback by either 
Cohort B PSTs of the use of online lectures, or by Cohort C PSTs of the delivery of F2F 
lectures. This lack of end of course commentary, regarding mode of delivery, is a critical 
point and provides further qualitative evidence that the mode of delivery is only one of the 
determining factors in the success or otherwise of the courses. The question therefore 
remains - If not mode of delivery or team personnel, what factors contributed to the success 
of the course over a five-year period? 


Factors Impacting on Student Success 


The major themes that emerged from the data collected from the four cohorts were: a) 
course structure and dialogue; b) MCK and PCK of lecturers; and c) rapport with the lecturer 
and with the discipline of mathematics. 


Structure and Dialogue 


One possible explanation for the success of the three courses was careful attention to 
structure and dialogue, as informed by Transactional Distance Theory (TDT). Moore and 
Kearsley (1993) suggests that TDT accounts for the psychological and communications 
space that occurs between learners, which is shaped by the learning environment and by the 
patterns of activity of individuals within the environment. TDT is influenced by two core 
factors: the structure of the program and the dialogue that exists between the teacher and the 
learner. These both impact on the level of autonomy required by each individual PST to 
successfully complete the course. Structure and dialogue can be manipulated to cater for, in 
this case, PSTs studying mathematics education via various modes of course delivery. 
Structure refers to the extent to which an educational program, or course within a program, 
can be responsive to the learning needs of individual PSTs. Dialogue refers to the interplay 
of words and actions between teacher and learner, and learner and learner, when one gives 
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instruction and the other responds. Much of the SET feedback across the four cohorts 
indicated that the courses were successful because of their structure (e.g. the course 
management system [Blackboard in all cases] and the availability of digital mathematics 
resources; and secondly on how dialogue was managed within the courses (e.g. F2F 
interactions [either physical or virtual or both], email correspondence, or course 
announcements. Of course, the mode of lecture delivery is one component of the overall 
course structure; however, as noted earlier, mode of delivery was barely mentioned in the 
end of semester feedback. Instead, the majority of feedback focused on the well-organised 
structure of the course as well as the following two factors. 


Lecturer Content and Pedagogy Knowledge 


There is a clear expectation from professional mathematics bodies that, upon graduation, 
PSTs are knowledgeable about best practice in mathematics education, including knowledge 
of students, knowledge of mathematics, and knowledge of students' learning of mathematics 
(Frid, Goos, & Sparrow, 2008/2009). Given these expectations, a key teaching goal is 
ensuring that the PSTs are competent and confident mathematics teachers. It is thus pleasing 
to see reflected in the SET scores of 4.9 or 5.0 (Table 3), an acknowledgement of our MCK 
and MPK. PSTs typically commented positively on the interconnected nature of our 
mathematics knowledge “Very knowledgeable about all content and teaches in a very 
interesting way to cater for all learners” {SET2015}; “Current and up to date understanding 
of mathematics education — great for confidence in the classroom” {SET2015}; and 
“Teacher displayed high level of confidence in her content knowledge and was able to 
explain mathematical concepts, theories and ideas at both very basic and complex levels to 
ensure student understanding” {SET2017}. 


Knowing PSTs as Individuals 


Although our focus in these courses is, by design, mathematics education; our teaching 
of mathematics exists within the broader framework of training PSTs to be primary school 
teachers, not solely specialist primary school mathematics teachers. The third piece of the 
“success puzzle” therefore transcends just mathematics education and instead reflects the 
fact that our teaching philosophy, regardless of mode of delivery, is highly relational i.e. it 
recognises that learning can only occur when a positive learning relationship has been 
established between the learners and the lecturers. Therefore, we always prioritise the 
building of rapport and PST engagement. PST feedback suggest that these endeavours were 
successful - “Very interesting, clear, approachable and builds a rapport with students, 
engaging, knowledgeable, understanding and helpful” {SET2015}; “XXX positive attitude 
towards mathematics and passion for learning mathematics radiates and encourages 
students to learn” {SET2016}; and “XXX is a highly approachable, enthusiastic tutor, who 
genuinely wants the best for every student that she teachers” {SET2017}. 


Conclusion 


Based on our five-year experience of delivery of mathematics education across two 
universities, we are confident that the mode of delivery is not the critical factor in the overall 
success of the courses. This is an important contribution given that the views of many 
mathematics educators often focus on the mode of delivery as the determining factor in the 
success or otherwise of the mathematics education courses they teach. Based on the data in 
this project, this is clearly not the case. A second observation is that success is often seen by 
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educators as a consequence of what might be termed “the great teacher” construct. This has 
been the experience of Author 2 who, in presenting previously on the issue of mode of 
delivery, has been challenged by colleagues that the success of the course is due to the 
teacher and not due to the structure of the course guided by TDT principles. Whilst we 
acknowledge that a specific teacher can play an important role in the effectiveness of 
courses; as this course has been successful, with different lecturers, we are confident that the 
teacher is not the sole determining factor in course success. Rather than relying on the mode 
of delivery or the specific personnel to explain success, the data suggests that it is more 
important for academics, when planning and delivering PST mathematics education courses 
to: focus on course design in terms of how the course is structured and the mode of delivery; 
ensure high personal levels of MCK and PCK and ensure students are engaged in the 
mathematical content (especially considering that many undergraduates commence 
mathematics education courses with a deficit view); and finally to commit to establishing 
and maintaining student rapport both with the teaching team and also with the discipline of 
mathematics to provide maximum opportunities for student learning. It is perhaps 
convenient to explain teaching success as dependent on external factors (mode of delivery, 
time allocations, etc.) or internal factors (i.e. teacher charisma) that are both difficult to 
change; however, we suggest that considerations of course structure, dialogue, teacher 
knowledge and building student relationships are much more important, and critically, are 
within the power of all academics to encompass in their mathematics education courses. 
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